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Optimal  Exit  Probabilities  and  Differential  Games 

by 

Wendell  H.  Fleming  and  Chun-Ping  Tsai 


* 


Abstract :  The  problem  is  to  control  the  drift  of  a  Markov  diffusion 
process  in  such  a  way  that  the  probability  that  the  process  exits 
from  a  given  region  D  during  a  given  finite  time  interval  is 
minimum.  An  asymptotic  formula  for  the  minimum  exit  probability 
when  the  process  is  nearly  deterministic  is  given.  This  formula 
involves  the  lower  value  of  an  associated  differential  game.  It  is 
related  to  a  result  of  Ventsel  and  Freidlin  for  nearly  deterministic, 
uncontrolled  diffusions. 
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OPTIMAL  EXIT  PROBABILITIES  AND  DIFFERENTIAL  GAMES 


Wendell  H.  Fleming  and  Chun-Ping  Tsai 

1.  Introduction.  Let  £  be  an  n-dimensional  stochastic  process 
with  continuous  sample  paths,  defined  for  times  t  >  0.  Let  D 
be  a  given  bounded  open  subset  of  n-dimensional  En.  If  £  (0)  e  D, 
the  exit  time  is  the  first  time  t  such  that  £  (x )  e  3D.  For 
fixed  T  >  0  the  exit  probability  is  PC*  <  T) . 

In  this  paper,  we  suppose  that  £  is  a  controlled  process, 
which  obeys  the  stochastic  differential  equation 

Cl.l)  d£  =  b[S(t),ytt)]dt  +  e1/2dw, 

where  y(t)  is  a  control  applied  at  time  t,  e  > 
and  wft)  an  n-dimensional  brownian  motion.  We 
yCt)  e  Y,  where  Y  £  Em  is  a  given  compact  set. 
control  processes  y  admitted  in  (1*1)  have  the 

y  Ct)  =  yCt.CCt)) 

where  y  is  any  Borel  measurable  function  from  [0,T]  x  En 

into  Y.  As  initial  data  we  have  5C0)  *  x,  for  given  vector 

x  e  D.  For  general  background  on  these  concepts  see  [FR1,  Chap.  VI). 

Let  us  denote  the  exit  probability  by  t^,  to  indicate 
dependence  on  the  feedback  control  law  jr  and  on  e.  Let 
qy  *  PC*y  <  T).  We  seek  a  control  law  which  minimizes  the  exit 


0  a  parameter, 
assume  that 
Moreover,  the 
feedback  form 
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probability  q^.  From  the  point  of  view  of  applications,  this  is 
a  reasonable  criterion  of  performance  of  the  control  system,  if  D 
is  considered  as  a  region  of  states  in  which  the  system  operates  in 
an  "acceptable"  way.  The  minimum  exit  probability  is  denoted  by 


Cl- 2)  qE  =  min  q^. 

L  L 

In  §2  we  shall  replace  the  initial  time  0  by  any  initial  time  s, 

0  <  s  <  T.  Then  q  *  q  (s,x)  is  a  function  of  the  initial  time 

0 

s  and  initial  state  x  »  5(s).  The  function  q  satisfies  the 

dynamic  programming  equation  with  suitable  boundary  conditions 

(see  (2.4),  (2.5)  below).  In  special  cases,  this  boundary  value 

problem  was  solved  numerically  by  Dorato  and  Van  Melaert  [DVM] . 

However,  it  is  generally  difficult  to  get  effective  information 
0 

about  q  and  the  optimal  control  law  in  this  way.  Instead,  we 

0 

seek  an  asymptotic  formula  for  q  valid  for  small  e.  Our  main 
results  (Theorems  4.1  and  4.2)  assert  that 

(1.3)  -lim  e  log  qe  *  I, 

e-t-0 

where  I  is  the  lower  value  of  a  certain  differential  game.  The 
proofs  involve  some  technical  complications.  However,  two 
different  heuristic  arguments  can  be  given  to  suggest  the  validity 
of  (1.3).  One c£  these  is  as  follows  (a  second  heuristic 
derivation  of  (1.3)  based  on  the  dynamic  programming  equation  for 
qe  is  given  in  §2.)  Given  a  feedback  control  y  the  drift  co¬ 
efficient  in' the  stochastic  differential  equation  (1.1)  is 
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(1-4)  b^Ct.O  =  ba,jr(t,5)). 

If  is  Lipschitz,  then  the  Ventcel-Freidlin  estimates 

[Fr  2,  Chap.  14]  imply 

Q'5)  -lim  e  log  q^  =  I  , 

e-vO  i-  2- 

T 

Cl. 6)  I  *  min  i  f  |b  Ct.Kt))  -  <Ht)|2dt, 

2-  L  J  0  2- 

and  where  &  is  the  class  of  all  <j>  e  C^CIO.TJjE11)  such  that 
KO)  =  x  and  <|>(e)  e  3D  for  some  0  €  [0,T],  A  stochastic 
control  proof  of  Cl .6)  is  given  in  [F4,  §7].  If  we  set 

Cl- 7)  iCt)  ■  zCt),  0  <  t  <  T, 

then  we  can  regard  zCt)  as  a  new  control  associated  with  the 
minimum  problem  Cl*6)>  Intuitively,  minimizing  corresponds 

in  the  limit  as  e  -*■  0  to  choosing  £  to  maximize  I  .  This 
leads  us  to  consider  a  differential  game,  described  formally  as 
follows.  Let 

Cl. 8)  LCx,y,z]  -  \  | b Cx , y )  *  z|2. 

In  Cl* 6)  we  may  replace  the  upper  limit  T  by  the  first  time  9 
such  that  <K9)  6  3D,  and  take  Ht)  -  b  Ct.fCO)  for  t  >  9. 
The  integral  in  Cl. 6)  is  then 
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r0 

Cl. 9)  L(<j>(t),y(t),z(t))dt, 

J0 

where  yCt)  *  y.(t,<Kt))  is  chosen  by  the  original  controller 
and  zCt)  is  chosen  by  a  new  second  controller  who  knows  The 

second  controller  then  knows  y(t)  as  well  as  <j>(t). 

In  the  formal  description  of  the  differential  game,  <|>(t)  is 
the  state  of  the  game  at  time  t,  with  initial  data  <j>(0)  =  x, 
x  £  D.  The  game  dynamics  are  (1.7),  subject  to  the  restriction 
that  HO  must  reach  3D  at  some  time  9  6  (0  ,T] .  The  original 
controller  seeks  to  maximize  the  payoff  (1.9),  and  the  second 
controller  seeks  to  minimize  (1.9).  Note  that  y(t)  appears 
in  (1.9)  but  not  in  (1.7).  Thus  the  role  of  the  maximizing 
controller  is  a  passive  one.  The  term  "lower  value"  refers  to 
the  advantage  in  information  which  the  minimizing  controller  has 
(see  [Fr  1] ) . 

The  intuitive  description  above  does  not  suffice  to  define 
rigorously  a  differential  game  with  lower  value  I.  In  the 
rigorous  treatment  we  shall  obtain  I  as  the  limit  of  values  of 
corresponding  discrete-time  games,  following  the  method  of  [FI). 

We  have  been  unable  to  obtain  the  limit  in  (1.3)  directly  from 
the  Ventcel-Freidlin  estimates  (1.5).  The  restriction  by 
Lipschitz  under  which  (1.5)  is  derived  is  too  restrictive  for 
the  problem  of  optimal  exit  probability;  and  there  is  no  guarantee 
of  uniformity  with  respect  to  £  of  the  limit  in  (1.5). 

The  outline  of  the  paper  is  as  follows.  In  §2  we  introduce 
the  function  Ie  *  -e  log  qe.  Note  that  (1.3)  asserts  that 
I  -*•  I  as  e  -*■  0.  This  logarithmic  transformation  changes  the 
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£ 

dynamic  programming  equation  for  q  into  the  nonlinear  parabolic 

partial  differential  equation  (2.7)  for  I£.  As  e  -*■  0 ,  this 

equation  degenerates  into  a  first-order  equation,  which  is  just 

the  Isaacs  equation  for  the  lower  value  I£  of  the  differential 

game  described  intuitively  above.  The  function  I£(s,x)  tends 

to  +"  as  s  -*■  T.  We  introduce  a  penalty  function  method,  in 

which  I  is  replaced  by  a  solution  IM  of  (2.7)  with  terminal 

£  £ 

data  M$>(x)  when  s  =  T.  As  M  -*•  <*>,  IM  increases  to  I  .  In 
§2  we  give  a  priori  estimates  for  I  ,  based  on  standard  estimates 
for  brownian  motion  and  on  the  maximum  principle  for  parabolic 
equations . 

In  §3  we  introduce  penalized  differential  games,  by  a 

discretization  procedure  similar  to  [FI],  [F2 ] .  For  technical 

reasons,  we  impose  a  bound  |z|  <  c  for  the  second  controller. 

e 

It  turns  out  that  for  sufficiently  large  c  >  c(M),  IM  is  the 
lower  value  of  the  penalized  stochastic  differential  game.  As 
e  -*■  0,  1^  -►  IM  where  1^  is  the  lower  value  of  the  corresponding 
penalized  deterministic  differential  game. 

As  M  -►  »,  IM  increases  to  a  limit  I.  In  §4  we  show  that 

£ 

I  -*■  I  as  e  -*■  0,  and  identify  I  as  the  lower  value  of  the 
differential  game  described  formally  above. 


2.  Preliminary  Results.  We  make  the  following  assumptions 

,n 


throughout  the  paper.  D 
boundary  3D  a  manifold  of  class  C 


is  a  bounded,  open  subset  of  E  ,  with 
3 


The  control  space  Y  is 


a  compact  subset  of  E 
The  vector  function  b 


n 


We  consider  times  in  the  interval  [0,T], 


in  (1.1)  is  of  class  C1(En 


Y;En). 


X 


Moreover,  for  some  constants 


I b  (x , y ) |  <  B1§ 

|bCx,y)  -  b Cx,y)  |  <  B2|x-x|. 

This  is  actually  no  restriction  since  b  is  C1  and  we  stop  the 
solution  5  to  Cl.l)  when  CCt)  leaves  the  bounded  set  D  or 
when  t  =  T.  The  function  L  in  Cl -8)  then  satisfies 

C2.1)  |  L  Cx ,  y ,  z )  -  LCx,y,z)|  <  AC1+I  z| )  |x-x| . 

C2.2)  cjzl2  -  c2  <  L, 

for  some  constants  A,  c ^  >  0,  c2- 

Let  3/  be  the  class  of  Borel  measurable  functions  y  from 
[0,T]  x  En  into  Y.  For  a  feedback  control  law  ye  S',  the 
stochastic  differential  equations  Cl*l)  take  the  form 

C2.3)  dC  -  bz(t,5(t))dt  +  e1/2dw 

with  b^  as  in  Cl *4).  We  consider  C2.3)  with  initial  data 
CCs)  *  x,  where  0  <  s  <  T,  x  €  D.  CLater,  we  specialize  by 
taking  s  -  0.)  This  is  a  problem  of  completely  observed  controlled 
diffusions  [FR1,  Chap.  VI].  However,  instead  of  the  kind  of 
performance  criterion  considered  in  [FR1J  we  wish  to  minimize  the 
exit  probability 


i  « 


q®(s,x)  =  P(t*  <  T), 


with  the  first  time  t  >  s  such  that  5 (t)  €  3D.  Let 


Q  =  (0,T)  *  D,  Q  -  [0 ,T]  x  D,  Q  =  Q  -  {T}  x  9D. 


Given  A  c  En  and  0  <  g  <  1  we  say  that  a  function  q(s,x) 
has  finite  g-norm  on  A  if  there  exists  a  constant  K  such  that 


|qCs,x)  -  q  (s '  ,  x  ’  )  |  <  K(|x-x'|6  +  |  s-s '  |  e/2) 


for  all  (s,x),  O'  ,x')  €  A.  We  say  that  q  €  C^’  (A)  if  q 


and  the  gradient  q^  in  the  variables  x  =  (x1,...^11)  have 
finite  g-norm  on  A.  If  q,q  ,qs,  q  i  i,  j  =  1 . n  have 

X  X  17 

finite  g-norm  on  A,  then  we  say  that  q  6  C:’  (A). 

P 

The  minimum  exit  probability  q£(s,x)  belongs  to  C*,2(A) 
for  any  compact  A  c  Qq  and  g  €  (0,1).  Moreover,  in  Qq  the 
following  dynamic  programming  equation  holds: 


(2.4) 


0  ■  +  |  A  qE  +  min  q®*b(x,y), 

six  y6Y  x 


where  Ax  is  the  Laplacean.  See  [FR1,  p.  161],  also  the  Remark 
below.  The  boundary  data  are 


(2.5) 


qE(s,x)  *  1  for  s  <  T,  x  6  3D;  qE(T,x)  *  0,  x  £  D. 


Let 
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(2.6)  I£(s,x)  -  -  e  log  qE(s,x). 

By  elementary  calculus  Ie  satisfies  the  nonlinear  parabolic 
partial  differential  equation 

(2.7)  0  -  Is  +  |  AxIe  +  H(x,Ix), 

(2.8)  H(x,p)  =  -  \  | p | 2  +  max  p-b(x,y). 

L  y£Y 

Here  p  =  (P|,...,p  )  denotes  a  row  vector. 

From  (2.5)  the  boundary  data  are 

(2.9)  I£(s,x)  =0  for  s  <  T,  x  6  3D;  IE(T,x)  »  +• ,  x  €  D. 

The  function  L  in  (1,8)  is  strictly  convex  and  quadratic 
as  a  function  of  z.  The  function  dual  to  L,  in  the  sense  of 
duality  for  concave  and  convex  functions,  is 

(2.10)  H(x,y,p)  =  min  [L(x,y,z)  +  p-z], 

z 

where  the  min  is  over  all  z  £  En.  An  easy  calculation  shows 
that 

=  -  \  I P 1 2  +  P*b(x,y), 


H(x,y,p) 
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and  therefore  by  (7.8),  (2.10) 

(2.11)  H(x,p)  =  max  min  [L(x,y,z)  +  p*z). 

y€Y  z 

Equation  (.2.7)  then  takes  the  form 

(2.7')  0  =  1^  +  j  A  I e  +  max  min  [L(x,y,z)  +  I^*z). 

S  Z  X  y€Y  z  x 

This  is  the  dynamic  programming  equation  associated  with  a 
stochastic  differential  game.  Similar  equations  were  considered 
in  [F2].  When  e  =  0,  (2.7*)  reduces  to  a  first  order  partial 
differential  equation,  which  is  the  Isaacs  equation  for  the  non¬ 
stochastic  differential  game  described  formally  in  §1.  This 
provides  a  second  heuristic  argument  for  considering  this 
differential  game,  in  addition  to  the  heuristic  argument  given 
in  §  1 . 

We  shall  approximate  the  infinite  terminal  condition 

£ 

I  (T,x)  =  +«  by  a  large,  but  finite,  terminal  condition  as 

? 

follows.  Let  4>(x)  be  a  function  of  class  C  (15)  satisfying 

(2.12)  Yj  dist(x,3D)  <  $(x)  <  Y2<list(x,3D)  for  x  €  D 

$(x)  =0  for  x  t  D, 

where  are  positive  constants  and  dist(x,3D)  is  the 

distance  between  a  point  x  and  the  boundary  3D.  For  each 
M  >  0  let 
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(2.13) 


qnCs,x) 


M$[C(iyAT)j 

min  E  exp{- - — £ - 

1 


where  as  before  §  satisfies  (2.3)  with  initial  data  5(s)  =  x 

C  £ 

and  T^A  ^  =  min[T^.,T],  The  minimum  is  taken  among  all  feedback 

controls  y  £  This  is  a  stochastic  control  problem  of  the 

kind  considered  in  [FR1,  Chap.  VI,  §4,6].  According  to  results 
£ 

given  there,  q^  satisfies  the  dynamic  programming  equation  (2.4). 

The  function  q^  belongs  to  Cg’^(Q)  H  Cg’^(A)  for  any  compact 

_  £ 

A  c  Q0  and  6  €  (0,1).  Moreover,  qM(s,x)  is  also  the  minimum 

of  the  expectation  in  (2.13),  taken  among  nonanticipative 

Y-valued  control  processes  [FR1,  pp.  162-163], 


Remark.  By  estimates  for  parabolic  equations,  and  the  fact 

that  q^  is  uniformly  bounded,  the  partial  derivatives 
£  £  £  •• 

(qm)s»(Qm)  ^,(q^)  i  j  satisfy  Holder  conditions  on  any  compact 

subset  of  Q0  which  are  uniform  with  respect  to  M  for  fixed 
e  >  0.  See  [LSU] ,  also  [FR1,  Appendix  E] .  The  boundary  data  are 

q^(s,x)  «  1,  for  s  <  T,  x  6  3D;  q^(T,x)  =  exp[-  ] ,  x  6  D. 

£  £  ~  £  £ 

Moreover,  0  <  q^+^  5  q^  <  1 •  The  limit  q  (s,x)  of  q^(s,x)  as 

1  2 

M  »  is  a  solution  of  (2.4)  belonging  to  C^’  (A)  for  any  compact 
_  ~  £ 

A  c  Moreover,  q  has  the  boundary  data  (2.5).  A  Verification 

Theorem  [FR1,  p.  159]  implies  that  qe  *  qe  and  that  qe(s,x)  is 
the  minimum  exit  probability.  Actually,  the  proof  of  the 
Verification  Theorem  in  [FRlj  must  be  modified  slightly  since 
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£ 

q  is  discontinuous  at  points  (T,x),  x  6  3D.  This  causes  no 
difficulty,  since  the  random  variable  S (T)  is  absolutely 
continuous  with  respect  to  Lebesgue  measure  in  En.  Hence 


P(te  =  T)  <  P(£(T)  6  3D)  =  0. 


Let 

(2.14)  Ij^Cs.x)  =  -e  log  q^(s,x). 

Since  q^  >  q£,  and  q^  4  q£  as  M 

(2.15)  I*  <  I£,  I^j  t  I£  as  M  °°* 

Moreover,  I£  is  a  solution  of  (2.7)  (or  equivalently  (2.7')) 
with  the  boundary  data 

(2.9m)  I^(s,x)  =  0  for  s  <  T,  x  €  3D;  I^(T,x)  *  M$(x),  x  6  D. 

£  £ 

Let  us  now  give  some  bounds  for  I  and  1^.  In  these 
£  £ 

lemmas  we  write  I  (s,x;T),  I^(s,x;T)  to  emphasize  dependence 
on  the  final  time  T.  Since  H(x,p)  does  not  depend  on  time,  we  have 

(2.16)  Ie(s,x;T)  =  ie(0,x;T-s),  I*(s,x;T)  =  I£(0 ,x;T-s) . 

Lemma  2.1.  Let  T'  <  T.  There  exists  U  such  that 
ie(s,x;T)  <  U  for  0  <  s  <  T* ,  x  €  D,  and  for  small  e.  The 
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constant  U  depends  on  upper  and  lower  bounds  for  T  -  T' ,  but 
not  on  e. 

Proof.  By  (2.16)  we  may  take  s  =  0  and  impose  the  bounds 
0  <  k  <  T  <  K.  Since  |b|  <  B^,  (2.3)  implies 

51(T)  -  x1  >‘B1T  +  e1/2(w1(T)  -  w1(0)) 

where  x1  is  the  first  component  of  x  etc.  Since  D  is 

bounded,  there  exists  a  constant  a  such  that  £*(T)  -  x^  >  a 
'  £ 

implies  t  <  T.  Hence 
1  - 

PCT'^VCT)  >  (eT)"1/2(a+B1T))  <  P(r£  <  T)  =  . 

-1/2  1 

Since  T  '  w  (T)  has  a  standard  normal  distribution,  we  have 
[Fel,  p.  166] 

lim  X*2log  PCT'^w1^)  >  X)  =  -  i  . 

X-M»  '  1 

Hence,  for  small  e  we  have  for  all  y_ 

e  (a+B,T) ^ 

'e  log  q^  <  - T -  . 

£  £ 

By  (1.2),  I  *  -e  log  q  satisfies  the  same  inequality.  In 
Lemma  2.1  let  U  =  k  ^(a+BjK)2. 
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Lemma  2.2.  Let  T'  <  T.  There  exists  a  positive  constant 

£ 

Mq  (depending  on  T'  but  not  on  e)  such  that  I  (s,x;T)  < 
Mpdist(x,3D)  for  0  <  s  <  T’  and  for  small  e. 

Proof.  By  Lemma  2.1  it  suffices  to  verify  such  an  estimate 
in  a  neighborhood  of  any  £  3D.  Let  be  the  exterior  unit 

normal  to  3D  at  x^;  and  let 

Dj  =  (x  £  D:  distCx.x^)  <  P} 


where  P  is  to  be  chosen  later.  By  (2.7*) 


I 


e 

s 


+  2 


e  V‘ 


+  max  L(x,y, z. ) 
y£Y  1 


*  Vzi 


>  o. 


The  maximum  occurs  at  some  y*(s,x),  measurable  in  (s,x) 
[FR1,  p.  199]. 

Choose  any  T"  €  (T’,T).  For  s  <  T",  x  €  D,  define  the 
process  by 


d>|>  =  zxdt  +  e1/2dw,  ^(s)  =  x. 


Let  6e  =  min(T",  exit  time  of  ’j'(t)  from  D} . 

From  Ito's  formula 

f9e 

IE(s ,x;T)  <  E  j  LC<Kt),y*,z1)dt  +  I£(e£,<K9£)  ;T) 
where  y*  =  y* (t Ct) )  •  Moreover,  Ie(T",x;T)  <  U  by  Lemma  2.1, 


-14- 


“"d  1  Cs,x;T)  =o  for  x  €  3D.  Since  L(x,y\Zl)  is 

by  some  C* 

I£(s,x;T)  <  C*E(ee-s)  +  uP(ee=T"), 

P(ee=T”)  =  P(Q  s>T"  -  s )  <  (T"-s)-1E(9e-s). 


To  prove  Lemma  2.2  it  then  suffices  to  show  that 


(2,17J  E(e£-S)  <  c  dist(x,  3D) 

x  6  Dj,  0  <  s  <  T'.  This  is  done  as  in  proving  [F3 
as  follows. 

Let  9£  *  min(T-,  exit  time  of  *(t)  from  D^) ,  and 

S  Cs.xJ  =  E(9e-s3,  g1(s,x)  «  ECQj-s). 

Then  ge,g^  both  satisfy  the  parabolic  equation 

^2*18)  u  +  j  A  u  +  u  -z.  +  l  =  o 

in  [0,T  )  x  Dj  with  ge  =  g£  =  0  for  x  €  3D  or  s  =  ' 
x  €  3D^  -  3D,  gj  =  0,  0  <  ge  <  x. 

Let 

<Kt)  =  x  +  z1(t-s) 


bounded 


Lemma  4.2], 


For 
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and  8^  the  exit  time  of  <j>(t)  from  D.  We  choose 

P  <  j  (T-T")  and  small  enough  that  6®  -  s  <  2  dist(x,3D)  for 
all  x  £  Dj.  The  function  g^(s,x)  =  6®  -  s  satisfies  in 
[0,T")  x  Dj  the  first  order  equation 

(2.19)  g°s  +  g®.zx  +  1  =  0, 

with  g°  =  0  for  x  £  3D  and  >  0  for  x  €  3D^  -  3D  or 
s  =  T".  Let  u  >  0  and  g  =  (l+u)g°.  By  (2.18)  and  (2.19),  for 
small  e 

*s  *  f  4x*  *  VZ1  *  1  s 

Hence,  by  the  maximum  principle  for  parabolic  equations  g^  <  g 
on  [0,T"J  *  Dj.  By  Ito's  formula 

61 

g£(s,x)  =  E  j  dt  +  Ege(ej,^(0^)). 

Since  the  first  term  on  the  right  is  g^(s,x)  and  ge  <  T, 

(2.20)  ge(s,x)  <  g  (s ,  x)  +  TP(«KeJ)  €  3Dj  -  3D). 

By  reasoning  as  in  [F4,  p.  488],  for  s  <  T',  x  6  Dj 


P0K«J)  €  3D  -  3DX)  <  eBgJ(s,x) 
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£  £ 

for  some  B.  Since  <  g,  (2.20)  implies 

g£(s,x)  5  T^BT  g°Cs,x)  <  {-ill1  dist(x.aD). 

For  small  e,  we  then  have  (2.17)  with  C  =  3(l+p).  This  proves 
Lemma  2.2. 

Lemma  2.3.  For  T*  <  T,  there  exists  a  constant  such 

that  le(s,x;T)  <  I^(s,x;T')  for  all  M  >  ,  0  <  s  <  T',  x  €  D, 

and  for  small  e. 

Proof.  Both  Ie(s,x;T)  and  I^(s,x;T')  satisfy  (2.7). 

By  Lemma  2.2,  for  small  e 

I E(T ' ;x,T)  <  MQdist(x,3D) , 

and  by  C2.12) 

I^(T' ,x;T' )  =  M*(x)  >  My^dist (x, 3D) . 

For  x  6  3D,  IE  *  1^  =  0.  Let  *  mqy1^*  the  maximum 

£  E 

principle  for  parabolic  equations,  I  <  IM  in  the  cylinder 
Q'  *  (0,T')  x  D  for  any  M  >  .  This  proves  Lemma  2.3. 

The  minimum  in  (2.11)  is  taken  among  all  z  €  En.  For 
technical  reasons  we  wish  to  consider  equations  corresponding  to 
(2.7)  when  bounds  are  imposed  on  |z|. 
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For  c  >  1  let 


Zc  =  (z  €  En:  | z |  <  c}. 


Let 


Me 


be  the  solution  in  Q  of 


(2.21) 


(IMc}s 


+  J  A  if.  +  HC(x,  (I^  )  )  =  0, 
2  x  Me  ^  K  Mc'x'  ’ 


(2.22) 


Hc(x,p)  =  max  min  [L(x,y,z)  +  p*z] 
y£Y  z£Zc 


with  the  same  boundary  data  (2.9M)  as  for  IM- 


The  solution  I 


Me 


belongs  to  Cg,1(Q)  n  Cg’i'(A)  for  any  compact  A  c  Qq  and 
3  €  (0,1).  Since  Hc  >  H,  we  always  have  I^c  >  1^.  In 
section  3  we  will  show  that  for  c  large  enough  (c  >  c(M)), 


a, 2, 


=  1 


jyJq •  See  (^3,15). 


Lemma  2.4.  There  exists  a  constant  B  =  B(M)  such  that 

£ 

lMcCs»x)  <  B  dist(x,8D),  for  all  (s,x)  €  Q  and  for  small  e. 

Proof.  Since  3D  is  of  class  C3,  D  has  the  exterior 
sphere  property.  Following  [F5,  p.  275],  given  Xj^  €  3D  let 
A  be  an  n-dimensional  spherical  ball  with  center  X£,  radius 
1x^X2!  intersecting  D  in  the  single  point  x^.  Let 
V(x)  *  IX-X2I  -  jx^-X2l  and  let  £(x)  =  -  Tx(x).  For  small  e 
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I  V  ♦  z.*x  < 


1 

2 


for  all 


£  £ 
J  - 


x  €  D.  As  in  the  proof  of  Lemma  2.2  the  function 
satisfies 


Js  +  ?  V£  +  Jx'i  *  L*Cx)  *  °» 

L*(x)  s  max  L(x,y,£(x)). 
y£Y 

Moreover,  L*(x)  <  L^,  for  some  L^.  For  B  >  2L^ 
CBY-Je)s+  |  Ax(B'F-J£)  +  (B4'-J£)x-z  5  0. 


For  x  e  3D,  BY (x)  -  Je(s,x)  =  BY (x)  >  0.  For  B  >  B (M)  >  2Lj 
we  have 


BY  (x)  -  JECT,x)  =  BY(x)  -  M4>(x)  >  0, 

for  all  x.  €  D.  The  maximum  principle  for  parabolic  equations 
implies  BY  -  JE  >  0  in  Q. 

Given  x  €  D,  let  x^  be  a  point  of  3D  nearest  x.  Then 
dist(x,3D)  »  Y(x),  which  implies  Lemma  2.4. 


3.  Penalized  Differential  Games.  In  the  differential  game 
formally  described  in  §1,  the  minimizing  controller  is  required 
to  bring  the  game  state  <f>(t)  to  3D  by  some  time  9  <  T.  We 


now  omit  that  restriction  and  instead  impose  a  penalty  if  3D  is 
not  reached.  Let  us  begin  by  describing  formally  the  penalized 
differential  game,  and  the  corresponding  penalized  stochastic 
differential  game  for  e  >  0.  The  penalized  game  has  the  same 
dynamics  (1.7)  as  before,  but  instead  of  (1.9)  the  payoff  is 

re 

(3.1)  L[<j>(t)  ,y(t)  ,z(t)]dt  +  M$>  [<J>  (6  )  ]  , 

Jo 

where  now  0  =  min{T,  exit  time  of  <fr(t)  from  D} .  The  penalty 
M$(0)  =  0  if  0  <  T  by  (2.12).  For  e  >  0  the  game  state 
Y(t)  obeys  a  white-noise  perturbation  of  (1.7): 

dt  =  z  (t)dt  +  e1/,2dw. 


The  payoff  is 


;9  e 

(3.2)  E^l  L[¥(t),y(t),z(t)]dt  +  M*[ne  )] 

0 


with  6  =  min{T,  exit  time  of  'Kt)  from  D) .  For  technical 
reasons,  we  impose  a  bound  |z(t)|  <  c  for  the  minimizing  controller. 
Moreover,  we  replace  the  initial  time  0  in  (3.1)  or  (3.2)  by 
any  initial  time  s  £  [ 0 , T ] ,  and  consider  the  initial  data 
<j>  (s)  =  ip(s)  =  x.  The  Isaac's  equation  for  the  penalized  stochastic 
differential  game  is  (2.21);  for  the  deterministic  game  the 
Isaac's  equation  is  the  first-order  equation  corresponding  to 
(2.21)  when  e  »  0.  We  shall  show  below  that  I^c  is  the  lower 
value  (in  a  suitably  defined  sense)  of  the  penalized  stochastic 


E 


differential  game  and  that  its  limit  IMc  as  e  0  is  the 

lower  value  of  the  penalized  deterministic  game.  We  also  show  that 
£  £ 

*Mc  *  *M  ^or  *ar2e  enough  c  >  c(M). 

In  order  to  treat  these  games  on  a  rigorous  basis,  we 
discretize  the  game  dynamics  and  payoff  in  the  same  way  as  [FI], 

[F2 ] .  A  somewhat  different  discretization  procedure  was  used  in 
[Frl] . 

In  the  discussion  to  follow  we  fix  M,c,  and  T.  For 
N  =  1,2,. . .  let 

S  -  R  •  *k  *  •  k  ■  1 . N  *  !• 

For  simplicity  we  do  not  indicate  the  dependence  of  6  and  t^ 
on  N.  Let  n£,  k  =  1,...,N,  i  =  l,...,n,  be  mutually  random 
variables  which  assume  the  values  1  and  -1  each  with  probability 

2  .  In  order  to  avoid  certain  analytical  questions  of  integrability 
and  the  existence  of  a  value,  we  shall  arrange  that  each  of  these 
games  is  finite.  Let 


Y.  e  Y,  c  ...  c  Y,  Z.  c  Z7  c  ...  e  E  , 

X  b  X  M 


where  YN,Z^  are  finite  sets  for  each  N,  the  union  of  the  sets 

I 

Yn  is  dense  in  Y,  and  the  union  of  the  sets  is  dense  in  E 

For  c  >  1  let 


n  ZM. 


Zc  =  {z:  |z|  <  c}, 
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We  shall  define  for  initial  data  (s,x)  with  s  =  t^, 

A  =  1,...,N,  x  €  D,  a  game  with  2(N-A+1)  moves.  At  move  2k  -  1, 
the  first  controller  chooses  some  yk  6  Y^,  and  at  move  2k  the 
second  controller  chooses  some  zk  €  Z^,  k  *  A,...,N.  Both 
choices  are  obtained  using  strategies,  as  described  below.  The 
state  ^+1  after  move  2k  is  determined  from  the  system  of 
difference  equations 

C3.3)  Vi  *  *  «1/2e1/2V  '  *• 


At  each  move  2k  -  1  or  2k  the  controllers  know  all  previous 

moves  and  also  n£ » • • • ♦  1 •  T^e  game  stops  at  the  first  step  Fe 

when  either  .  £  D  or  F£  =  N,  We  call  t  *  (Fe-1)<5  the 

F  +1  F 

exit  time  for  ipj,.  After  play  stops,  the  first  controller  receives 
the  payoff 


C3.4) 


*  (s,x)  =  Es 


1  6L^k,yk,zk)  +  ***(*  e  Y 


|k=£ 


F  +1 


When  e  =  0  we  consider  the  corresponding  deterministic 
game  whose  states  <j>k  obey 


(3.5) 


^k+1  ^k  +  6zk»  ^  A  =  x» 


with  payoff 


^(s,x)  =  l  5L(<l>k,yk,zk)  +  M$(«j>F+1) 
k—  A 


(3.6) 
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where  tp  is  the  exit  time  for  4*^. 

The  game  is  described  more  precisely  by  introducing  the  idea 
of  strategy  (also  called  a  6-strategy).  For  the  deterministic 
game  a  strategy  for  the  first  controller  is  the  (N-£+l) -vector 
T  =  (rA,...,rN)  such  that  is  any  point  y£  6  YN  and  rk, 

k  =  l  +  is  any  function  of  zm>  i  <  m  <  k  taking 

values  in  Y^.  A  strategy  for  the  second  controller  is  the 
(N-A+l) -vector  A  =  (A^,...,AN)  such  that  A^,  k  =  £,...,N,  is 


any  function  of 


Q 

y  ,y,  ,  Z  <  m  <  k  with  values  in  ZVI.  For 
1  m  1  k  ’  N 


the  stochastic  game  (e  >  0)  strategies  for  both  players  are 

k  k 

defined  similarly  except  that  the  functions  T  ,A  ,  k  =  l  +  1 , . . . ,N , 
can  depend  on  n  ,  m  *  A  , . . .  ,k  -  1  too.  A  strategy  T  for  the 
deterministic  game  defines  a  strategy  for  the  stochastic  game  in 
the  obvious  way.  On  the  other  hand,  given  the  random  inputs 

£  =  (n^ . nN)  a  strategy  T  for  the  stochastic  game  induces 

a  strategy  (denoted  by  r/C)  for  the  deterministic  game.  Similar 
remarks  apply  to  the  second  controller.  Given  a  pair  of 
strategies  (r,A),  and  a  vector  of  random  inputs  t  in  the 
stochastic  game,  the  successive  moves  are  found  by  taking  y^  at 
move  2)1-1  and  setting 


h  =  a* cyt ),  yl+1  *  rA+1(yltn4), 


Let  us  indicate  explicitly  dependence  on  the  strategies  r,A  by 

£ 

writing  the  payoff  in  (3.4)  as  w  (s,x,r,A).  Similarly,  we  write 
n(s,x,r,A)  in  (3.6)  when  e  ■  0.  Note  that  the  game  is  biased 


"<rr* 


'  t-  ■**rVJ**- 
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in  favor  of  the  minimizing  controller,  who  knows  y^  before 
choosing  z^.  The  concept  of  strategy  used  in  [ F 2 ]  is  slightly 

different,  but  equivalent  to  the  present  one. 

Given  the  initial  data  (s,x)  and  N,  each  of  these  games 
has  a  finite  number  of  possible  positions.  There  are  a  finite 
number  of  control  choices  for  each  controller  at  each  move.  From 
the  theory  of  positional  games,  the  stochastic  difference  game  has 
a  value  W^(s,x)  and  optimal  strategies  F*  ,A*  exist: 

Tre(s,x,r,A*)  <  W^(s,x)  5  ite(s,x,r*,A) 

£  & 

for  every  r  ,A ,  with  equality  when  r  =  T  ,  A  =  A  .  In  fact, 
WN(s,x)  can  be  defined  by  backward  induction  by  the  functional 
equation 

C3.7)  W^fs,x)  =  max  min  E[<5L(x,y,z)  +  W^(s+<5,i/>)] ,  s  =  t^  <  T,  x  €  D, 

YN  ZN 

C3.8a)  Wj^Cs.x)  ~  0,  x  i  D 

C3.8b)  *Jj(T,x)  -  M<t (x ) , 

where 

^  =  x  +  6z  + 

and  the  components  p1  of  p  are  independent  random  variables, 
each  with  values  ±1  with  probability  j  . 
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P 


'  ^+*&lL-4‘  U-  y  A, 


When  e  =  0  the  deterministic  difference  game  has  a  value 
WN(s,x)  satisfying  the  functional  equation  corresponding  to  (3.7) 
«snd  the  conditions  (3.8).  We  shall  show  that  as  N  ■+  e  -*■  0, 

WN  and  the  function  I^c  defined  at  the  end  of  §2  tend  to  the 
same  limit.  Moreover,  this  limit  IM  does  not  depend  on  c, 
for  large  enough  c.  See  Theorem  3.3  and  formula  (3.15). 

We  begin  with  a  series  of  lemmas  which  give  uniform  estimates 
for  W^  -  WN  and  for  W^(s,x')  -  wjjj(s,x). 


Lemma  3.1.  There  exist  constants  Cj  =  (M) and  such 

that  for  N  >  and  small  e 

wn(s>x)  5  C1dist(x,8D)  +  C1(6+n(6e)1/Z),  s  =  t£,  £  =  1,...,N,  x  £  D. 


Proof.  Given  x^  £  D,  let  x^  be  a  point  of  8D  nearest 

Xq.  Choose  X2  and  T(x)  =  | x - x 2 |  *  | x ^ - x 2  I  as  in  the  proof  of 

Lemma  2.4.  Let  the  minimizing  controller  use  the  control  z£  chosen 

as  follows.  Given  N,  divide  =  { | z |  <  1}  into  nonoverlapping 

Borel  measurable  sets  A^,...,AV  such  that  each  A^  contains  exactly 

one  point  z*  €  Z?,  and  max  diam  A.  -*•  0  as  N  -*•  00 .  Let 

i<i<v  1 

^(x)  =  z1  if  -Tx(x)  €  A^.  Since  |Tx|  =  1  there  exists  Nj  such 


that 


z(x)-'?x(x)  <  -  j,  x  £  D,  N  >  Nj. 

A  it  C 

take  z^  =  £(4'jc)  ;  since  we  always  take  c  >  1,  z^  €  Z^.  In  (3.3) 


^k  +  6zk  +  (6e)1/2rik- 


y*%— 
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Let  the  maximizing  controller  choose  yk  to  maximize 
and  let 


LOk»y>z£)  > 


M 


k  =  'V'V  •  VW'  if  *k  *  D' 


where  4  is  chosen  below.  If  (  D  (and  hence  k  >  F  )  we  let 


M  =  M  =  C  HI'  ). 
k  F  +1  1  F£+l 


For  <K  €  D,  k  <  N,  we  have  by  (3.7)  and  choice  of  y. 


W 


N^k’^k-1  -  Et6L^k,yk’zk:)  +  WN(tk+6^k+l)  I  Ck] 


5  54  *  E[w^(V6,Vl)Kk], 

where  |L(x,y,zOO  |  5  Lj^  and  4  =  (\ » •  •  •  > •  By  expanding 
H'OJ'k+i)  by  Taylor's  formula  about 

E[T0Pk+i)Kk3  =  n*k)  +  6zk’W  ♦  0((6+(6e)1/2)2). 

Since  z(x)-’t'  (x)  <  -  i  for  all  x  €  D,  we  have  for  small  e 

A  -  2 

and  <|>k  €  D 

WWl'kl  5  »»k)  ‘  7  • 

6C. 

EtMk.iiik)  5  «k  -  ~r  *  5  Mk 
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<  Cj.  If  ^  C  D>  then  =  M^.  Therefore, 

^^k+ll^kl  -  ^k  ’  w^ich  implies  by  iterating 

E(mn+i)  5  Mr 

Choose  Cj^  >  4L^  large  enough  that  C-^Cx)  -  M$(x)  >  0 
If  FG  =  N  and  ^N+1  €  D, 


on  D. 
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*N+1  =  Cl'1,^N+l^  '  -  0 


Otherwise , 


«N+i  =  VC*  E  )  >  c,[fCv  E  )  -  fC*  .)] 

1  1  Fe+1  1  Fe+1  Fe 


since  *  €  D  and  f(x)  >0  on  D.  Since  | f  |  =  1  we  have 

F  *  x 

in  the  second  case 


Vl  -  -Cll*  e  -  *  el  >  -Cj(6  +  n( 6e)1/2) 
F  + 1  F 


Since  =  G^Cxq)  -  WE(s,xQ), 


-C^fi+nCefi)1/2)  <  CjTCXq)  -  w{j(s,x0) 


Since  ^Cxq)  -  dist(Xg,3D)  and  Xq  6  D  is  arbitrary,  this  proves 
Lemma  3.1. 


Remark.  Similarly,  when  e  =  0,  an  estimate 
wNCs,x)  <  Cj [dist (x , 3D)  +  6]  holds. 


Lemma  3.2.  There  exists  C2  depending  on  M  and  c  such 
that  for  small  e  and  N  > 

|Wfl(s,x)  -  WN(s,x)|  <  C2[e1/2CT-s)1/Z  +  6  +  (6e)1/2J, 


s  =  t .  ,  l  =  1,...,N,  x  €  D. 
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Proof.  Given  (s,x)  let  T  be  an  optimal  strategy  for  the 
maximizing  controller  in  the  stochastic  game  and  A  an  optimal 
strategy  for  the  minimizing  controller  in  the  deterministic  game. 
Let  G  =  min(Fe,F)  with  F£,F  as  in  (3.4),  (3.6).  The  strategy 
A  is  played  against  F  in  the  stochastic  game,  giving  by 
dynamic  programming  arguments 


(*) 


G 

W£(s,x)  <  E{  l  6L0J-k 


,yk’V 


WNftG+l 


G+l 


)}. 


Given  a  vector  5  =  (n^ , . . . , nN)  of  random  inputs  to  (3.3),  the 
strategy  T/C  is  played  against  A  in  the  deterministic  game. 
We  have 


(**)  WN(s,x)  >  kI^6L(^k,yk,zk)  +  WN(tG+i^G+i). 

for  each  £.  Hence,  the  same  inequality  holds  if  we  take  the 
expectation  of  the  right  side. 

By  (3.3),  (3.5) 


¥ 


k+1 


*k+l 


where 


w, 


k 

l 


m 


By  (2.1),  L(*,y,z)  is  Lipschitz  with  some  constant  ,  for 
| z |  <  c.  Moreover,  $  is  Lipschitz  with  some  constant 
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Let  | | w | |  =  max{ | WjJ  :  k  =  A,...,N+1}.  Then 

Q 

J/lLWk’>’k-2k)  -  ^Wk’l  5  t1CT-s)e1/2||«||, 

s  mV1/2H«II- 

We  subtract  the  expectation  of  (**)  from  (*),  and  recall  that 
WN^G+1 ’^G+l^  -  0  together  with  (3.8): 

wJCs.x)  -  WN(s,x)  <  (KjCT-s)  +  MK2)e1/2E| |w| | 

+  E{WN(tG+i  ,<|<g+i) ;  G  <  N,  ^G+1  £  D) . 

Now  G  <  N,  ipG+1  e  D  imply  $G+1  C  D,  and 

distal, 3D)  <  |*(5n-*G.l|  ;  e1/2IM|. 

By  Lemma  3.1  we  have 

E^WN^tG*-l  ’^G+l^  ’  G  <  N»  ^g+1  €  D* 

<  C1(e1/2E| |w| |  +  6  +  n(6e)1/2). 

Since  the  sequence  of  components  w£,  i  =  1 . n,  form  a 

martingale  with  wj _1  =  0  we  have  [D,  p.  311] 

(3.9)  E||w||  <  2nE|wj|  <  2n(E| wj| 2)1/2  =  2n(T-s)1/2. 
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Therefore,  for  some  constant  C2 

W^Cs.x)  -  WN(s,x)  <  C2e1/2CT-s)  +  C 2(6  +  (6e)1/Z). 

The  opposite  inequality  is  proved  in  the  same  way,  by  taking  r 
optimal  for  the  deterministic  game  and  A  optimal  for  the 
stochastic  game,  and  using  the  Remark  after  Lemma  3.1. 

Lemma  3.3.  There  exists  a  constant  R  depending  on  M, 
such  that  for  small  e  and  N  > 

|wjj(s,x)  -  wjj(s,x*)|  <  R|  x-x*  |  +  R(6  ♦  (<5e)1/2), 

s  =  t^ ,  H  -  1,...,N,  x,  x'  €  D. 

Proof.  Let  T  be  an  optimal  strategy  for  the  maximizing 
controller,  for  initial  data  (s,x);  and  let  A  be  an  optimal 
strategy  for  the  minimizing  controller,  for  initial  data  (s,x'), 
in  the  stochastic  game.  We  use  the  strategies  r,A  both  in  the 
game  with  initial  data  (s,x)  and  in  the  game  with  initial  data 
(s,x').  Let  be  the  solutions  to  (3.3)  with 

^(s)  *  x,  Y^(s)  =  x'.  [Recall  that  strategies  are  expressed  in 
terms  of  past  control  choices  and  random  inputs,  but  not  in  terms 
of  the  states  ^ ]  Let  G  *  min(Fe,Fe  )  where  Fe,Fe  are  the 
corresponding  stopping  times  in  (3.4).  Then 
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wn(s-x>  <  B,kUWk.rk.,k,  *  wjjctG+1,*G+1)>, 

WN(S’X,)  ?  E'XSL(,'k'1'k'Ik)  *  WN(tG*l-"W>- 

By  (3.3),  ^  =  x  -  x' .  We  then  have  as  in  the  proof  of 

Lemma  3 . 2 

e  e  G 

(*)  WN(s,x)  -  WN(s,x')  <  E  kI^6|L(^k,yk,zk)  -  L(*£,yk,zk)| 

+  MK2|x-x'|  +  E{W*(tG+1,«G+1);  G  <  N,  U»G+1  €  D). 

The  terms  on  the  right  side  of  (*)  are  estimated  as  follows. 
(2.1)  and  (2.2),  for  some 

lL(*k’>Vzk>  -  L«,k’>Vzk>l  :  V1  *  L<*k’>Vzk»l’‘  -  *'l 

Since  A  is  optimal  for  initial  data  (s,x'), 

G  E 

E  E  6L(*k,yk,zk)  -  wn(s»x')* 

If  G  <  N,  i|»G+1  €  D,  then  i/>£+1  C  D  and 

dist  (^G+ 2 , 3D)  <  I^g+i'^q+]_I  ®  lx_x'l* 

On  the  right  side  of  (*),  we  have: 


By 


1st  term  <  Aj (T-s+W^Cs ,x’ ) ) | x-x' | . 
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By  Lemma  3.1  j 

j 

3rd  term  <  Cj|x'-x|  +  • 


Since  dist(x,3D)  <  diam  D  <  ",  another  application  of  Lemma  3.1 
gives  a  bound  on  W^(s,x’ ) .  Hence,  for  some  constant  R 

W^(s,x)  -  W^(s.x')  <  R| x-x ' |  +  R(6+(6e)1/2). 

Since  the  roles  of  x  and  x'  can  be  exchanged,  this  proves 
Lemma  3.3. 

£  £ 

We  will  next  show  that,  for  fixed  e,  WN[s,x)  tends  to  1^ 

as  N  -*■  ".  The  main  step  in  doing  this  is  Lemma  3.5.  To  make 

the  backward  induction  in  that  Lemma,  we  use  the  following  slightly 

different  payoff  for  the  discrete- time  stochastic  difference  game. 

Let  Dp  be  the  P-neighborhood  of  D.  Let  Je  be  an  extension 
£ 

of  IMc  to  a  cylinder  [0,TJ  *  D  p,  for  some  p  >  0,  as  follows: 
jE(s’X)  -  imc(s’x)’  if  x  € 

JE(s,x1  +  rz(x1))  =  Je(s,x1-rz(x1))  +  ZrJ^Cs.Xj) 

£ 

for  Xj  €  3D,  z(Xj)  the  exterior  unit  normal  to  D  at  Xj,Jv 
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the  normal  derivative,  0  <  r  <  P.  The  function  Je  belongs  to 
Cg^ao.T]  x  Dp)  and  to  ’ 2 ( [0 ,T' ]  x  Dp)  for  any  T’  <  T  and 
0  €  (0,1).  Moreover,  if  distfx.gD)  <  p. 


C3 . 1 1 3  |J£(s,x)|  <  SP 


where  S  »  3B  with  B  =  B(M)  as  in  Lemma  2.4.  Instead  of  (3.4) 
we  take  the  payoff 


ttE(s,x) 


F 

E{  1  6L^k»yk 
k=*  K  K 


2k> 


J  (t 


Fe+1 


»V  e  )). 
F  +1 


Let  ^(s>x)  denote  the  value  of  the  stochastic  difference  game 

with  payoff  fie(s,x),  x  e  D.  For  x  e  Dp  -  D  let  W^(s,x)  = 

Je(s,x).  If  Fe  =  N  and  yN+1  €  0,  the  last  term  is  MK^+j) 

as  in  (3.4).  Otherwise,  ^  £  D,  t/,  C  D.  By  (3.3)  and  the 

Fe  Fe+1 

fact  that  |nj|  =  1,  |zk|  <  c, 


provided 

(3.12) 


dist  (<P  _ 

Fe+1 

N  >  Nx 


,3D)  <  |V  . 

F  +1 

where  Nj  is 
c6  ♦  n^V'2 


-  *  el  <  P 

Fe 

large  enough  that 

<  P. 


By  (2.12)  t(jp  )  a  0  if  ^  (  D.  By  (3,11)  we  have 

Fe+1  Fe+1  r 

for  N  >  |ire-ne|  <  Sp  for  every  pair  of  strategies  r,A. 


Hence 
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(3,13)  |W^(s,x)  -  (s ,  x)  |  <  SP, 

for  all  s  ■  t  ,  l  =  1 , . « . ,N,  x  6  D. 

For  fixed  e  and  N  let 

C3.14)  q  =  sup | Ww(t. ,x)  -  JECtf>x)|. 

*  x€D  N  *  *’ 

•V  £ 

By  choice  of  n  ,  ^n+i  =  0*  In  Lemmas  3.4  and  3.5,  M,c,e  are 
fixed.  The  constants  appearing  in  those  lemmas  may  depend  on 
M, c , £ . 


Lemma  3.4.  There  exist  S^,N2  such  that 

|W*(s,x)  -  *(x)|  5  SjCT-s)1/2  +  SXP 


for  N  >  N^,  0  <  T  -  s  <  1,  x€D. 


Proof.  Consider  any  pair  of  strategies  r,A  for  a 
stochastic  game  with  initial  data  s  =  t^ ,  x  =  'P^.  We  have,  for 
some  B,  0  <  L  <  B  on  Q  *  Y  *  Zc.  Choose  N2  such  that  (3.12) 
holds  for  N  >  N  Then 


|*£(s,x)  -  <f>  (x)  |  <  B(T-s)  +  ME  |  $  (’I'  )  -  *(x)| 

F  +1 


E|J  Ct  e  A  E  )  -  M*0  )| 

F  +1  F  +1  F  +1 


The  last  term  is  Q  if  Fe  =  N,  €  D;  otherwise  *0  )  =  0 

N  1  F  +1 
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and  distO  _  ,3D)  <  P  by  (3.12).  By  (3.3),  (3.9)  and  the 

F  +1 

fact  that  | |  <  c , 

E| ip  - x |  <  c(T-s)  +  2ne1/2(T-s)1/2. 

F  +1 

Let  K  be  a  Lipschitz  constant  for  $.  Then,  for  0  <  T  -  s  <  1, 
|iie(s,x)  -  4> (x)  |  <  B(T-s)  +  MK(c+2ne1/2)  (T-s)  +  SP 
with  S  as  in  (3.11).  We  take 

S1  =  max[B+MK(c+npe1/2) ,  S]  . 

Since  the  above  inequality  holds  for  any  r ,A  we  get  Lemma  3.4. 

Since  Je  is  Holder  continuous  on  Q,  |Je(s,*)  -  JE(T,*)| 
is  uniformly  small  on  D  if  T  -  s  is  small.  Since  Je(T,x)  = 

4>(x)  for  x  €  D,  from  Lemma  3.4  we  have  the: 

Corollary.  Given  a  >  0  there  exist  k  >  0  and  ^  such  that 
q^<a  rf  0<T-s<k,  s=t^,N>  N£. 

Lemma  3.5.  Given  T'  <  T  there  exist  constants  A  >  0, 

0  <  o  <  j  such  that  the  following  is  true:  given  y  >  0 
there  exists  Nq  such  that  for  N  >  Ng 

q^  <  (y+A6a) (T1 -s)  ♦  qm 


for  s  =  t.  <  t  <  T* . 
-  z  -  m  - 
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Proof.  The  function  JE  is  of  class  *  Dp) 

for  any  3  €  (0.1)-  Let  a  *  3/2.  As  above  we  take  N  >  N 2 

1  n 

large  enough  that,  for  any  x  £  D,  | z|  <  c,  n  =  (n  , . . . ,n  )  with 

n*  =  ±1. 


♦ 


x  +  6z  + 


61/2c1/2 


n 


belongs  to  Dp.  By  Taylor's  formula 

Je(s  +  6  ,<fO  =  JE(s  ,ip )  +  <SJE(s,^)  +§>!, 

Je(s,^)  =  Je(s,x)  +  JE(s  ,x)  •  O-x) 

+  j  l  Ji  x  (s,x)(*i-xi)C^-xj)  +  ®2, 
L  iJ=1  XiXj 

where  for  suitable  constants  A^,A2 

|®il  5  Ai6l+a»  l®Zl  ^  A2^-x|2  +  2a,  3  =  2a. 


Since 


|JE(s,<jO  -  JE(s,x)|  <  A3|ip -x|  201  , 


JeCs+6,<|»)  -  Je(s,x)  ♦  6Jg(s,x)  +  J*(s,x)-(#-x) 

+  T  l  JE  _  Cs,x)0l^i-xi)0l'j-xj)  ♦  @-f 

2  i,j-l  xixj  3 


we  have 


I®,|  5  a  (61+a+6(^-x|2a  +  |^-x|2  +  2a) 


By  taking  expectations,  we  have 


EJe(s+6,^)  =  J£(s,x)  +  i[Jg(s,x)  +  J£(s,x)-z 


*  |  ve>  *  ®. 


2  n 


(h)  =  E®  *  L-  I  z1z;,Je 
w  T  i,j-i  xixj 


_  C  E 

Now  L  is  uniformly  continuous  on  D  *  Y  *  Z  and  Jx  is 


uniformly  continuous  on  Q.  By  (2.22),  given  u  >  0  there 


exist  Nq  >  and  a  function  y*  from  Q  into  the  finite  set 


Ym  ,  such  that 


L(x,y* (s ,x) , z)  +  J£(s,x)-z  >  Hc (x, J£ (s ,x) )  -  u 


for  all  (s,x)  €  Q,  |z|  <  c. 
If  Y  <  3 


|*-x|Y  =  | 6z  +  e46^n| 1  <  A5 ( 6 T  +  (e4<54|n|)') 


1  1 


1  1 

Y  .  ,.7x2 i „ i n  Y, 


1  Y 

and  E| n  |  =  1,  i  =  l,...,n.  We  have 


E|(h)j  |  <  Ag [ i1+Ct  ♦  «l*2“  *  5Z+2“], 


For  NQ  large  enough,  6  <  1  and 


I  ©I  1  A6] 


for  suitable  A. 


£  £ 

Since  J  =  I„  is  a  solution  of  (2.21)  in  Q,  we  have  for 


x  €  D,  y*  =  y*(s,x) 


E{  6L(x,y* ,  z)  +  Je(s  +  6,'i<)}  >  JE(s,x)  -  6p  -  A61+<\ 


Let  us  now  estimate  in  terms  of  q^  +  1.  By  (3.14) 


J  (s  +  6>)  <  q^  +  1  +  Wn(s+6>) 


if  V  e  D.  This  inequality  holds  also  for  ip  C  D,  since  J£  = 


by  definition  in  that  case.  Hence,  for  all  z  €  Z( 


E{  6L  (x  ,y* ,  z)  +  WN(s  +  6,ij;)}  >  J  (s,x)  -  6p  -  A61+“  -  q£  +  r 


Since  WN  satisfies  the  functional  equation  (3.7),  this  implies 


wj(s,x)  >  JE(s,x)  -  <5p  -  A61+a 


ql  +  l' 


A  similar  argument,  choosing  z*  =  z*(s,x,y)  with  values  in 


z!x  such  that 
0 


L(x,y ,z*)  +  J^-z*  <  Hc(x,J^(s,x))  +  P 


I 
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for  all  (s,x)  €  Q,  y  €  Y,  shows  that 

W^(s,x)  <  Je(s,x)  +  6y  +  A61+u  +  q£  +  J. 


Therefore , 


q£  <  6(y +A6a)  +  ql  +  1. 

Since  6(m-£)  =  t  -  t^  <  T’  -  s,  this  proves  Lemma  3.5. 

By  (3.12)  the  number  P  in  (3.13)  can  be  chosen  arbitrarily 
small  if  N  is  large  enough.  By  (3.13),  Lemma  3.5,  and  the 
Corollary  to  Lemma  3.4  we  have: 

~  £ 

Theorem  3.1.  For  fixed  e,WN(s,x)  -  IMc(s,x)  and 

£ 

WN(s,x)  -  IMc(s,x)  tend  to  0  as  N  -*■  <»,  uniformly  for  x  €  D, 
s  =  t^,  £  =  1, . . . ,N. 


£ 

As  in  [F2]  we  call  I^c  the  lower  value  of  the  penalized 

stochastic  differential  game,  for  initial  data  (s,x)  €  Q.  From 

Lemma  3.3  and  Theorem  3.1  we  have  the  following  uniform  Lipschitz 
£ 

condition  for  ^c^5’’^' 

Theorem  3.2.  There  exists  a  constant  R  =  R(M) ,  such  that 


lIMc(s,xJ  '  IMc.(s’x')l  -  Rlx_x'l 


x ,x'  €  D,  0  <  s  <  T 


for  small  e 
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Theorem  3.2  implies  the  a  priori  estimate  on  the  gradient 

ldMc)xl  5  *• 

By  (1*8)  the  minimum  over  z  €  En  of  L  +  p*z  is  attained 
at  z  =  b  +  p.  Since  |b(x,y)|  5  B^,  |p|  <  R  implies 
|b+p|  <  B1  +  R.  Hence,  by  (2.11),  (2.22) 

Hc(x,p)  =  H(x,p) ,  | p  j  <  R,  c  >  Ej  +  R. 

Let  p  =  (Imc)x»  c(m)  =  Bi  +  RCM) •  By  (2.7),  (2.9M)  and  (2.21) 

(3.15)  I^c  -I®,  c  >  c (M) . 

Let  us  now  take  s  =  0.  Since  b  =  b(x,y)  the  problem  is 
autonomous,  and  hence  this  is  no  real  restriction  (see,  in 

particular  (2.16)).  Lemma  3.2  gives  a  uniform  estimate  for 

£  £  £ 

WN  -  W^,  which  tends  to  0  as  N  e  -*•  0.  Since  J  =  I^c 

on  Q  for  c  ^  c(M),  we  have 

£ 

Theorem  3.3.  As  e  -*■  0 ,  1^(0,  x)  tends  to  a  limit  IM(0,x). 

Moreover,  W^(0,x)  -  1^(0, x)  tends  to  0  aj;  N  -+•  °°,  uniformly 

for  x  E  D. 

As  in  [F2]  we  call  1^(0, x)  the  lower  value  of  the 

deterministic  penalized  game  starting  at  x  £  D.  As  in  §2  let  us 

also  denote  this  lower  value  by  1^(0, x;T)  to  indicate  dependence 
on  the  final  time  T.  By  (2.15),  1^  is  nondecreasing  in  M,  and 
by  Lemma  2,1,  1^  is  bounded  above  for  fixed  T  >  0.  Hence, 
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Im(0,x;T)  is  nondecreasing  in  M  and  bounded  above.  Let 


(3.16)  1(0, x;T)  =  lim  IM(0,x;T),  x  €  D. 

M+® 

G  £ 

4.  Main  Results.  In  this  section,  we  show  that  I  =  - e  log  q 
tends  to  a  limit  I  as  e  -*•  0,  and  interpret  I  as  the  lower 
value  of  the  differential  game  described  formally  in  §1.  In 
view  of  (2.16)  it  suffices  to  consider  initial  time  s  =  0 ,  as 
already  noted  at  the  end  of  §3.  The  discrete  time  games  in  §  3 
then  start  with  l  =  1 .  We  recall  that  the  value  WN(0,x)  of  the 
deterministic  penalized  game  in  §3  depends  also  on  M  and  c.  For 
T*  <  T,  let  a  =  T'T  *.  In  the  following  lemma,  W^(0,x)  denotes  the 
value  of  the  corresponding  game  in  which  T  is  replaced  by  T'  and  c 
by  a  *c . 

Lemma  4.1.  Let  j  T  <  T'  <  T.  There  exist  constants  Rj,N^,c* 
(depending  on  T)  such  that  JWN(0,x)  -  W^(0,x)|  <  R^(T-T’)  for  any 
M,  N  >  Nx,  c  >  c*. 

Proof.  Let  6  =  TN"*,  6'  =  a6  =  T’N  For  any  pair  of 
strategies  (T,A)  for  the  discrete  time  deterministic  game  in  3  with 
time-step  6,  we  consider  the  corresponding  strategies  (I”, A*)  for  a 
game  with  time-step  6' ,  in  such  a  way  that 

s  x,  y£  =  yk,  z£  =  a'1z]c,  k  =  1,...,N, 


where  denotes  the  state  and  y£,z£ 

latter  game.  By  (3.5)  we  have  =  $k 


the  controls  for  the 
for  all  k.  We  wish  to 


estimate  the  difference  of  the  respective  payoffs  By 

(1.8),  (3.6) 

"  -  '•  -  j  J  «((l-a)|bk|2  .  Cl-a‘1)Ut|2] 
k=l  *  K 

where  =  M^.y^).  Since  0  <  a  <  1,  the  first  term  in  the 
sum  is  positive,  and  the  second  term  is  negative.  Since  |b^[  <  B^, 

B1T(l-a). 

Since  this  is  true  for  any  r,A,  and  since  Ta  =  T' , 

(*)  WN(0,x)  <  W^(0,x)  ♦  \  B1(T-T'). 

On  the  other  hand,  from  (2.2),  (3.6),  and  4>  >  0 

^  J/l’kl2  ^  *  *  Tsj1  f"*c2T). 

wn(°.x)  5  WN(0,X)  +  (WN(0,X)  +  c2T). 

Let  us  show  that  WN(0,x)  <  K  for  N  .  Nj_ ,  c  >  c*  where  K,  Nx  and 
c*  depend  only  on  T.  For  this  purpose,  consider  any  zQ  E  Z^j  with 
| z 0 1  >  1/2.  Let  the  minimizing  controller  use  the  constant  control 
zk  =  C*ZQ,  where  ^  c*T  >  diam  D.  With  this  choice,  there  exists 
such  that  exit  occurs  by  step  F  <  N  for  N  >  N^.  Moreover,  the  payoff 
it  satisfies  *  <  <5FL*  <  TL*  where  L*  is  a  bound  for  L(x,y,z) 


-vy**v» 
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when  | z |  <  c*.  Let  K  =  TL*.  Since  T'  <  T  <  2T' 

(L*+c,) 

(**)  W'(0,x)  <  WN(0,x)  +  - —  (T-T') 

By  (*)  and  (**),  |W^(0,x)  -  WN(0,x)|  <  R^T-T')  where  R:  does  not 
depend  on  M.  This  proves  Lemma  4.1. 

We  have  by  Lemma  4.1,  Theorem  3.3,  and  (3.16): 

Corollary  4.1.  |lM(0,x;T')  -  I  (0,x;T)  |  <  R^T-T’) 

| I (0 ,x;T* )  -  I (0 ,x;T) |  <  R1(T-T’). 

By  combining  results  above  we  obtain  the  first  main  theorem. 


Theorem  4.1.  1(0, x;T)  =  lim  I  (0,x;T)  for  every  x  €  D 

e-*o 

and  T  >  0. 


Proof.  Consider  T'  <  T.  By  Lemma  2.3  and  Theorem  3.3, 
for  large  enough  M 


lim  sup  I  (0,x;T)  <  Iu(0,x;T'). 
e-*-0  '  M 


£  £ 

Since  IM  <  I  ,  for  each  M 


lim  inf  Ie(0,x;T)  >  lim  Iw(0,x;T)  =  IM(0,x;T) 
£-*•0  £-*0 


Hence  by  (3.16)  and  Corollary  4.1 
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lim  sup  Ie(0,x;T)  <  1(0, x;T)  +  R.  ( T-T') 

£-►0  ‘  1 

lim  inf  le(0,x;T)  >  1(0, x;T). 
e->0 

Since  T-T'  can  be  made  arbitrarily  small,  this  proves 
Theorem  4.1. 

It  remains  to  characterize  I  as  a  lower  value.  Let  us 
discretize  time  as  in  §3,  and  define  a  deterministic  game  as 
follows.  The  dynamics  are  again  (3.5),  and  strategies  T,A 
are  as  in  §3.  However,  instead  of  (3.6)  the  payoff  is  now 

F 

(4.1)  \,(o,x;r,A)  =  t  «L(*k,yk,zk)  +  X 

where  x  =  +0°  if  F  =  N  and  €  D  and  x  =  0  otherwise. 

Let 

4- 

(4.2)  vS(0 ,x ;T)  =  inf  sup  ^  (0 ,x; T , A) . 

N<C  r 

k 

The  notation  |A|  <_  c  means  |A  |  <  c  for  each  k;  this  is  the  same 
bound  imposed  in  §3.  We  now  write  WN(0,x;T)  for  the  value  WN(0,x) 
for  the  penalized  deterministic  game  in  §3;  recall  that  WN  depends 
also  on  M  and  c. 

Lemma  4.2.  Let  0  <  3  <  .  There  exist  c*,M^,Nj  (depending 

on  T  and  6)  and  a  constant  L^  such  that 

WN(0,x;T)  <  v{j(0,x;T)  <  W^(0,x;f)  +  L^T-T) 
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for  all  M  >  Mj,  t  =  N6,  Nj  <  f)  <  N,  T  -  T  >  BT,  c  >  c*. 


Proof.  The  payoff  n(=  nM)  in  (3,6)  satisfies  tf  <  7rro. 

Hence,  we  have  the  left-hand  inequality  in  Lemma  4.2.  To  obtain 
the  right-hand  inequality,  let  the  maximizing  controller  choose 
any  strategy  T.  We  define  a  strategy  A  for  the  minimizing 
controller  as  follows.  Let  A  be  an  optimal  strategy  for  the 
penalized  game  with  N  moves.  The  strategy  A  agrees  with  A 
on  moves  2k,  k  =  1,...,N.  Let  F  be  the  first  step  when 

A  A 

C  D  or  F  =  N.  If  C  D,  the  choices  z,  for 

F+l  F+l  K 

A  A  A 

k  =  N  +  1 , .  .  .  ,N  are  arbitrary.  If  F  =  N  and  €  D,  then 

N+l 

A 

N 

I  *L(*k»yk»zk)  +  m*(*a  )  <  Wa(0,x;T) 

k=l  K  N+l  N 

since  A  is  optimal.  Let  cj  *  4T'1diam  D.  For  c  >  cj,  the 
right  side  is  bounded,  as  shown  in  the  proof  of  Lemma  4.1.  By  (2.12) 
and  the  fact  that  L  >  0, 


dist^  , 3D)  <  CM 
N+l 


for  some  C.  Let  A  be  the  ^-neighborhood  of  3D,  for  suitable 


ci  <  3T .  We  choose  a  small  enough  that,  for  every  x  €  A,  there 


exists 

2 

l(x )  €  such  that 

x  +  s  z_(x)  $ 

D  for  some 

s  >  0,  s  <  o. 

Let  Mj 

>  Co"1.  When  F  =  N 

and  <f>^  6 

D,  then 

€  A  for  M  >  M 

In  this 

N+l 

N+l 

case,  let  z,  *  z(<p^ 

)  for  k  = 

N  +  1 . N. 

For  N  large, 

N+l 

the  first  F  when  <|>p  +  1  $  D 

satisfies 

F  <  N  since 

T  -  T  >  8T  >  a. 
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I£  <p^  $  D,  then  F  =  F.  Otherwise, 

F  +  l 


6(F-F)  <  6(N-N)  =  T  -  T. 


When  |z|  £  2,  L(x,y,z)|  <  for  some  L^.  Then 


"oo  =  l  +  &  L^k,yk’~ )) 

*c=l  N+l 


k=F+l 


£  WA (0 ,x;T)  ♦  L. (T-T) 
N  1 


For  this  strategy  A, 


sup  *„((), x;I\A)  <  WA(0,x;T)  +  L,  (T-T)  , 
T  N  1 


which  gives  Lemma  4.2. 


The  function  in  (4.2)  is  clearly  a  nonincreasing  function 


of  c.  Let  =  lim  V^.  Then 

c-*00 


(4.3) 


Vn(0,x;T)  =  inf  sup  ^(OjXjT.A) 

N  a  r 


where  the  strategy  A  for  the  minimizing  controller  is  now  chosen 

without  the  constraint  |A|  <  c.  Let  y^  =  (y^  *Y  2  *  •  •  •  denote  any 

sequence  of  control  choices  for  the  maximizing  controller,  y^  €  Y^. 

N 

Thus  y  is  an  open  loop  strategy.  Then 


, .  •w-'. 
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Vm(0 >x;T)  =  inf  sup  \,(0,x;yN  ,A) 


(4.4) 


|A|<c  yN 


Vn(0,x;T)  =  inf  sup  (0,x;y  ,A) 

N  A  yN 


To  see  this,  clearly  >  holds  in  (4.4).  On  the  other  hand,  any  pair 
of  strategies  r, A  and  initial  state  x  define  sequences  y,z  with 
T1  =  y ^  and 


*i  =  A'L(y1).  y2  =  1,2 (z^) »  z2  =  A2(y1»y2)» 


and  \,(0,x;y",A)  -  *.(0,x;r,A). 

We  use  the  following  truncation  procedure.  Let  ^ ;  ZN  -*■ 

be  such  that  wN(z)  ~  1  z  6  (recall  that  Z^  =  ZN  fl  {|  z  |  <  c). 

N 

Given  a  sequence  z  *  (Zj  , . . .  ,Zjj)  ,  let  z^  *  ^(z^).  Given  a  strategy 
A  -  (A1 . AN)  let  A  =  (A1 , . . . ,AN)  where  Ak  =  n  « °Ak.  As  in  (3.5) 


define  by 


♦k*i  ‘  *k  *  4V  *k.i  '  *k  *  sh>  *i  ’  fi  ’ 


For  1  <  F  <  N,  we  use  the  notation 


M  Ip  ■  max  Kl  • 

r  l<k<F+l  K 


Lemma  4.3.  Given  K  >  0  there  exists  B  =  B(K)  such  that 


I  6|ztJ^  <  K  implies  ||4>-<Hlr.  <  2Kc  and 
k«l  K  *  " 


I  6L^k,yk’^k^  -  ^6L^k,yk,zk^  +  Bc"  * 
k"l  k-1 

Proof.  Let  A  =  {k:  |z^|  >  c}  and  | A j  its  cardinality.  By 
Cauchy's  inequality 

c  <5 1 A  |  <  l  6|z,|  <  (6|A|)1/2K1/2, 

A  K 

and  therefore  6 | A |  <  Kc"2, 

l  6|zkl  <  l  6Ukl  5  Kc’1- 

A  K  A  K 
Since  IzJ  <  |zk| 

5  Cl  6 1 zk" 2k I ^ 2  -  6IAI  E  6lzk'zkl2 

A  A 

<  4 6 1 A | K  <  4K2c”2 . 

This  proves  the  first  assertion.  From  the  definition  (1.8)  of  L  and 

|ikl  -  lzkl’ 

F 

l  t6LC<*,k»yk,zki  *  6L(^k’yk’zk^ 

lc  1 

F  F 

>  \  l  «[!bkl2  -  |bk|2)  -  l  6(bkzk  -  bkzk) 
k*l  k-1 

where  bk  *  B($k,yk),  bk  =  M$k>yk).  Now  b(x,y)  is  bounded  and 

2 

Lipschitz  in  x.  Hence  |b|  is  also  Lipschitz  in  x,  and  for 
suitable  B, 


I  <5(jbk|  2  -  |6k|2)  >  -Bx|  Ip  >  -  2BjKc " 1 , 

k=l 


?  4bk<VJk>  5  B1  l  5'2k-Jkl  ?  2BlKc_1 

C“  1  A 

F  F 

j/Ck’V^  i  Bil !*-+! fp 

<  B1'2Kc‘1-T1^2-K1/2 


In  the  last  step  we  have  used  Cauchy's  inequality  and  |zk|  5  Ukl- 
By  combining  these  inequalities  we  get  Lemma  4.3. 

Lemma  4.4.  Let  8  >  0.  There  exist  N-^c^B  (depending  on  T 
and  8)  and  L^ ,  such  that 

V^(0,x;T)  <  Vn(0,x;T)  +  Be'1  +  L.BT 

n  w 

if  c  >  cx,  Nx  <  N  <  N,  T  -  N6,  f  -  T  >  BT. 

Proof.  By  (1.8)  given  K1  there  exists  K  such  that 

F  K  2 

l  6L0*Vyk,zk)  5  Ki  implies  l  6|zk|  <  K. 

k=l  k=l 

Let  Kj  >  VN(0,x;T),  and  consider  any  strategy  A  for  the  game  with 

N 

N  steps  (2N  moves)  such  that  for  any  open  loop  y  *  (y^  >  •  •  •  *yjSj) 


I  6L^k,yk,zk)  ■  "»(0»x;y  »A>  -  Kl 
k*l 


As  before  F  is  the  step  at  which  exit  occurs  (or  F  =  N) .  We  must  have 


<t>F+l  $  D  even  in  case  F  =  N,  since  otherwise  ^  =  «. 
with  $  steps  we  define  a  strategy  A  with  |A|  <  c 

^Ic  If 

1  <  k  <  F,  let  A  =  if^°A  be  defined  by  truncation, 
the  new  game  stops  after  F  <  F  steps.  Suppose  that 
4>f+|  $  D>  by  Lemma  4.3 


For  the  game 
as  follows.  For 
if  $F+1  i  D, 

4>P+P  €  D.  Since 


dist  (♦p  +  1 , 3D)  <  ||lf-'J>||F  5  2Kc 


As  in  the  proof  of  Lemma  4.2  let  A  be  the  a-neighborhood  of  3D, 
where  a  <  3T  is  sufficiently  small.  Then  4>F+F  e  A  for  large 
enough  c.  For  k  =  F  +  1,...,N,  let  zk  =  z.(^p+1)  ,  where  z(x)  is 
defined  as  in  the  proof  of  Lemma  4.2.  This  defines  the  strategy  A. 

For  large  enough  N.,  $  D  where  6(F-F)  <  3T.  For  each  open  loop 

N  F+1 

y  we  then  have 


,(0,x;yN,A) 


F 

=  l  <5L($k, 

k=l  K 


>vzk> 


F 

l 

k=F+l 


6L(<? 


k,7k’  — 


Let  L^  be  as  in  the  proof  of  Lemma  4.2.  By  Lemma  4.3 


7r„(0,x;yN,A)  <  l  <5L(*k,yk,zk)  +  Be'1  +  LpBT 
k- 1 


-1 


=  7rw(0,x;yN,A)  +  Be’1  +  L^T. 


Hence , 
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v5(0 ,x;f )  <  sup  7roo(0,x;yN,A)  < 

"  yN 

sup  »o#(0,x;yNtA)  +  Be'1  +  L,  BT . 

yN 

Since  the  inf  over  A  of  the  first  term  on  the  right  side  is 
Vn(0,x;T),  this  gives  Lemma  4.4. 

By  exactly  the  same  change  of  time  scale  argument  as  for  Lemma  4.1 

•V  jH* 

Lemma  4.5.  Let  T  <  T.  Then  there  exist  R^,Nj,c  such  that 

|vJ(0,x;T)  -  V^(0,x;T) |  <  RjCT-T) 

for  all  N  >  ,  c  >  c*. 

Theorem  4.2.  V^(0,x;T)  -*■  1(0, x;T)  as_  N  -*•  00 . 

Proof.  For  each  M  and  c  >  c(M)  Theorem  3.3  implies 

IM(0,x;T)  =  lim  WN(0,x;T)  <  lim  inf  vJi(0,x;T). 

M  N->®  N-*-00 

Let  3  >  0.  Choose  N,  T  *  N6  as  in  Lemma  4.4  such  that 

3T  <  T  -  T  <  2BT.  For  c  >  Cj,  N  >  Nj 

V^(0 ,  x;T)  <  VM(0 ,x;T)  +  Be'1  +  L.3T. 

N  '  N  1 

By  Lemma  4.5,  for  c  >  c* 


~lfrlr  . '  -  *— *  —  ■ 
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|  VC (0 , x ; T )  -  Vc(0,x;T)|  <  R1 (T-T)  <  2  3TR 
N  N  1 


Therefore , 


Im(0,x;T)  <  lim  inf  VN(0,x;T)  +  Be'1  +  3T(L  +2R,). 
N->°° 

However,  c  is  arbitrarily  large  and  3  arbitrarily  small. 
(3.16)  we  then  have 

1(0, x;T)  <  lim  inf  Vw(0,x;T). 

N->°°  w 

To  prove  the  opposite  inequality,  let  0  <  3  <  j  .  For 
we  can  choose  T  =  6ft  as  in  Lemma  4.2  with  3T  <  T  -  T  <  23T 

Vn(0,x;T)  <  vJj(0,x;T)  <  W  (0,x;T)  +  L^T-T). 

By  Lemma  4 . 1 

K(0,x;T)  -  W^(0,x;T)|  <  R.  (T-T) 

N  N  1 

“1  A 

provided  6  =  TN  is  small  enough  that  N  >  .  Since  T  - 

Theorem  3.3  implies 

lim  sup  Vw(0,x;T)  <  lim  W  (0,x;T)  +  23T(R.+L,) 

N  1  1 

-  Im(0,x;T)  +  23T(R1+L1) 

for  c  >  c(M).  Since  3  is  arbitrarily  small  and  1^  <  I , 


From 


N  large 
Then 


f  <  2  3T, 


4 


r 


lim  sup  Vn(0,x;T)  <  1(0, x;T). 
N-*-00 


This  proves  Theorem  4.2. 

Theorem  4.2  justifies  calling  I(0,x;T)  the  lower  value 


Concluding  remarks.  It  should  be  possible  to  show  that 
I(s,x;T)  =  I(0,x;T-s)  satisfies  for  almost  all  (s,x)  the 
Isaacs  equation,  which  is  the  first  order  equation  obtained 
from  (2.7')  when  e  =  0.  However,  we  have  not  done  so.  One 
could  perhaps  show  that  I  ( - , - ; T)  is  Lipschitz  for  s  <  T'  <  T, 
and  then  use  an  argument  like  the  proof  of  [F2,  p.  1005]. 

The  lower  value  I(0,x;T)  is  a  nonincreasing  function 
of  T.  Let 


I  (x)  =  lim  I (0 ,x;T) . 
T-m» 


It  would  be  interesting  to  investigate  various  properties  of 
I*(x).  For  instance,  is  1(0, x;T)  =  I*(x)  for  some  finite  T? 
That  kind  of  result  is  suggested  by  examples  treated  by  the 
method  of  characteristics  for  the  Isaacs  equation. 
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